The Performance of Group Lasso for Linear Regression of Grouped Variables

The lasso [19] and group lasso [23] are popular algorithms in the signal processing and statistics communities. In signal processing, these algorithms allow for efficient sparse approximations of arbitrary signals in overcomplete dictionaries. In statistics, they facilitate efficient variable selection and reliable regression under the linear model assumption. In both cases, there is now ample empirical evidence to suggest that an appropriately regularized group lasso can outperform the lasso whenever there is a natural grouping of the dictionary elements/regression variables in terms of their contributions to the observations [1, 23]. Our goal in this technical report is to analytically characterize the regression performance of the group lasso algorithm using `1/`2 regularization for the case in which one can have far more regression variables than observations. Analytical characterization of group lasso in this “underdetermined” setting has received some attention lately in the statistics literature [1, 14–17]. However, prior analytical work on the performance of group lasso either studies an asymptotic regime [1, 15–17], focuses on random design matrices [1, 16], and/or relies on metrics that are computationally expensive to evaluate [14, 15, 17]. Recently, Candés and Plan [4] successfully circumvented somewhat similar shortcomings